Craquelure as a Graph: Application of Image Processing and Graph Neural Networks to the Description of Fracture Patterns

Cracks on a painting is not a defect but an inimitable signature of an artwork which can be used for origin examination, aging monitoring, damage identification, and even forgery detection. This work presents the development of a new methodology and corresponding toolbox for the extraction and characterization of information from an image of a craquelure pattern. The proposed approach processes craquelure network as a graph. The graph representation captures the network structure via mutual organization of junctions and fractures. Furthermore, it is invariant to any geometrical distortions. At the same time, our tool extracts the properties of each node and edge individually, which allows to characterize the pattern statistically. We illustrate benefits from the graph representation and statistical features individually using novel Graph Neural Network and hand-crafted descriptors correspondingly. However, we also show that the best performance is achieved when both techniques are merged into one framework. We perform experiments on the dataset for paintings' origin classification and demonstrate that our approach outperforms existing techniques by a large margin.Comment: Published in ICCV 2019 Workshop

Artificial Color Constancy via GoogLeNet with Angular Loss Function

Color Constancy is the ability of the human visual system to perceive colors unchanged independently of the illumination. Giving a machine this feature will be beneficial in many fields where chromatic information is used. Particularly, it significantly improves scene understanding and object recognition. In this paper, we propose transfer learning-based algorithm, which has two main features: accuracy higher than many state-of-the-art algorithms and simplicity of implementation. Despite the fact that GoogLeNet was used in the experiments, given approach may be applied to any CNN. Additionally, we discuss design of a new loss function oriented specifically to this problem, and propose a few the most suitable options

Are all the frames equally important?

In this work, we address the problem of measuring and predicting temporal video saliency - a metric which defines the importance of a video frame for human attention. Unlike the conventional spatial saliency which defines the location of the salient regions within a frame (as it is done for still images), temporal saliency considers importance of a frame as a whole and may not exist apart from context. The proposed interface is an interactive cursor-based algorithm for collecting experimental data about temporal saliency. We collect the first human responses and perform their analysis. As a result, we show that qualitatively, the produced scores have very explicit meaning of the semantic changes in a frame, while quantitatively being highly correlated between all the observers. Apart from that, we show that the proposed tool can simultaneously collect fixations similar to the ones produced by eye-tracker in a more affordable way. Further, this approach may be used for creation of first temporal saliency datasets which will allow training computational predictive algorithms. The proposed interface does not rely on any special equipment, which allows to run it remotely and cover a wide audience.Comment: CHI'20 Late Breaking Work

Craquelure as a Graph: Application of Image Processing and Graph Neural Networks to the Description of Fracture Patterns

Cracks on a painting is not a defect but an inimitablesignature of an artwork which can be used for origin exam-ination, aging monitoring, damage identification, and evenforgery detection. This work presents the development of anew methodology and corresponding toolbox for the extrac-tion and characterization of information from an image of acraquelure pattern.The proposed approach processes craquelure network asa graph. The graph representation captures the networkstructure via mutual organization of junctions and frac-tures. Furthermore, it is invariant to any geometrical dis-tortions. At the same time, our tool extracts the propertiesof each node and edge individually, which allows to char-acterize the pattern statistically.We illustrate benefits from the graph representationand statistical features individually using novel GraphNeural Network and hand-crafted descriptors correspond-ingly. However, we also show that the best performance isachieved when both techniques are merged into one frame-work. We perform experiments on the dataset for paintingsorigin classification and demonstrate that our approachoutperforms existing techniques by a large margin